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APPEAL BRIEF 

Sir: 

This Appeal Brief under 37 C.F.R. §1.192 is submitted in support of the 
Notice of Appeal filed May 17, 2006, appealing to the Board from the action of 
the Patent Examiner's final Office Action, mailed March 28, 2006, finally 
rejecting pending claims 1-18 of the above referenced application. 
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I, REAL PARTY IN INTEREST 



The real party in interest of the instant application is Joseph Carrabis, 
sole inventor and a resident in the State of New Hampshire. 
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II. RELATED APPEALS AND INTERFERENCES 



There are no related appeals or interferences. 
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111. STATUS OF THE CLAIMS 



Claims 1-18 are pending in the application. Claims 1-18 stand finally 
rejected under 35 U.S.C. §1 03(a). The Applicant hereby appeals the foregoing 
final rejection for claims 1-18. 
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IV. STATUS OF THE AMENDMENTS 

One office action response was filed after the final Office Action. A 
response to an August 6, 2004 non-final office action was filed without 
amendment to the claims. A response to a February 14, 2005 non-final office 
action was filed without amendment to the claims. The final rejection was 
rescinded after filing an appeal brief. A response to an October 18, 2005 non- 
final office action was filed without amendment to the claims. In fact, the claims 
have not been amended since the original February 7, 2002 filing. 

Accordingly, the claims attached hereto in Appendix A reflect the office 
action responses mentioned above. 
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V. SUMMARY OF CLAIMED SUBJECT MATTER 

Ever start talking with someone (e.g., a spouse, a child, an employee, 
etc.) and feel like not a single word was being heard? They unintentionally, but 
completely, zoned you out. Or had a class where you had trouble 
understanding anything the professor said, while classmates did not encounter 
the same problem. Ever read the same page of a book four or five times and 
simply be unable to retain anything you read? It may be that at that time you 
were internally receptive to one mode of information, audible, for instance, but 
that book was providing the information in a visual mode. Or you may have 
been excited about something imminent, which internally left you wanting to 
receive information kinesthetically, even though most days you do prefer to 
receive information visually. 

The present invention, claimed as described below, allows a 
programmable device to determine an individual's preferred modality based on 
sensed psychomotor behavioral elements of the individual. The programmable 
device, as claimed in the dependent claims, may then be used to tailor a 
presentation of information or environment to the modality(ies) in which the user 
is most receptive. 

The Application has two independent claims, namely, claim 1 and claim 

12. 

Independent claim 1 is for a method of obtaining information regarding 
an environment for an individual using a programmable device. The individual 
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may have preferred modalities and engaged in activity. The first step of the 
method is sensing at least one psychomotor behavioral element of the activity 
engaged in by the individual. The second step of the method is determining the 
preferred modalities of the individual based on the psychomotor behavioral 
element of the activity engaged in by the individual. Subject matter of claim 1 is 
disclosed at least in the first paragraph (lines 1-15) on p. 23 of the Application 
and FIG. 1. 

Independent claim 12 is for a programmable apparatus for obtaining 
information regarding an environment to an individual having preferred 
modalities. The apparatus is made of at least one sensor for sensing 
psychomotor behavioral activity of the individual and a processing unit 
connected to the sensor. The processing unit receives the sensed 
psychomotor behavioral activity and calculates the preferred modalities of the 
individual based on the sensed psychomotor behavioral activity. Subject matter 
of claim 12 is disclosed at least in the first full paragraph (lines 2-9) on p. 27 of 
the Application and FIG. 3. 



VI. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 



The issue in this appeal is whether claims 1-18 are patentable over the 
cited references, with regards to 35 U.S.C. § 103(a). 



HAYES SOLOWAY P.C. 
130 W. CUSHING ST. 
TUCSON, AZ 85701 
TEL. 520.882.7623 
FAX. 520.882.7643 

175 CANAL STREET 
MANCHESTER, NH 03101 
TEL. 603.668.1400 

FAX. 603.668.8567 9 



VII. ARGUMENT 



HAYES SOLOWAY P.C. 
130 W. CUSHING ST. 
TUCSON, AZ 85701 
TEL. 520.882.7623 
FAX. 520.882.7643 

175 CANAL STREET 
MANCHESTER, NH 03101 
TEL. 603.668.1400 
FAX. 603.668.8567 



A. Case Law of U.S.C. 5103(a) 

It is well established at law that, for a proper rejection of a claim under 35 
U.S.C. §103 as being obvious based upon a combination of references, the 
cited combination of references must teach, disclose, or suggest, either 
implicitly or explicitly, all elements/features/steps of the claim at issue. See, 
e.g.. In re Dow Chemical . 5 U.S.P.Q. 2d 1529, 1531 (Fed. Cir. 1988), and In re 
Keller . 208 U.S.P.Q. 871, 881 (C.C.P.A. 1981). 

To establish prima facie obviousness of a claimed invention, all the claim 
limitations must be taught or suggested by the prior art. In re Royka, 490 F.2d 
981, 180 USPQ 580 (CCPA 1974). "All words in a claim must be considered in 
judging the patentability of that claim against the prior art." In re Wilson, 424 
F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). If an independent claim is 
nonobvious under 35 U.S.C. §103, then any claim depending therefrom is 
nonobvious. In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988). 

B. Claims 1. 11. 12 and 18 

1. US Patent No. 5.987.415 to Breese in View of U.S. Patent No. 

6.230.111 to Mizokawa Fail to Teach. Disclose, or Suggest 

Each Element of the Claims 
On page 3 of the final Office Action, claims 1,11,12 and 18 are rejected 
under 35 U.S.C. §103 as allegedly being obvious over U.S. Patent No. 
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5,987,415 to Breese et al. (hereinafter "Breese") in view of U.S. Patent No. 
6,230,1 14 to Mizokawa (hereinafter "Mizokawa"). 



2. Independent Claims 1 and 12 

Applicant respectfully submits that independent claims 1 and 12 are 
distinguishable over the present combination of references in that independent 
claims 1 and 12 recite features and steps that are neither disclosed, taught, nor 
suggested by the prior art of record. Independent claims 1 and 12 are set forth 
below. 

Independent claim 1 provides as follows: 

1 . A method of obtaining information 
regarding an environment for an individual, 
having preferred modalities and engaged in 
activity, using a programmable device, said 
method comprising the steps of: 

sensing at least one psychomotor 
behavioral element of the activity engaged by 
the individual; and 

determining the preferred modalities of 
the individual based on the psychomotor 
behavioral element of the activity engaged by 
the individual. 



Independent claim 12 provides as follows: 

1 2. A programmable apparatus for obtaining 
information regarding an environment to an 
individual having preferred modalities, said 
apparatus comprising: 

at least one sensor for sensing 
psychomotor behavioral activity of the individual; 
and 

a processing unit connected to the 
sensor for receiving the sensed psychomotor 
behavioral activity and calculating the 
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individual's preferred modalities based on the 
sensed psychomotor behavioral activity. 



3. The Examiner's Rejection 

It is stated in the final office action, on pages 3-5, that: 

As to claims 1, 8, 12, and 18 Breese teaches a 
system which including 'method of obtaining information 
regarding an environment for an individual having preferred 
modalities and engaged in activity using a programmable 
device' [see Abstract, col 4, line 57-62, col 6, line 36-44, col 
8, line 8-12, col 10, line 23-27, fig 3], environment for an 
individual corresponds to Breese's computer user interface 
that including observing user behavior, particularly user 
behavior caused from emotion and personality state as 
detailed in col 4, line 59-60, further it is noted that Breese 
specifically teaches a model that influence of emotion and 
personality based on Bayesian network as detailed in fig 3, 
programmable device corresponds to program modules 
being executed by a personal computer because program 
modules include processes, programs and like as detailed 
in col 6, line 40-42; 'sensing at least one psychomotor 
behavioral element of the activity engaged by the 
individual' [coi 8, line 23-28, line 35-41, col 10, line 34-42], 
Breese specifically teaches Bayesian net integrate various 
aspects of emotion and personality in a single model for 
example cognitive response as well as physical response 
based on the mental state, further, it is noted that 
personality nodes capture or senses individual emotional 
and or personality state as detailed in col 10, line 34-42; 
'psychomotor behavior element on the activity engaged by 
the individual' [col 12, line 49-59, fig 6], psychomotor 
behavior elements corresponds to loud, angry voice tones, 
calm quite voice related to emotional state and personality 
of the user as detailed in col 12, line 50-52. It is however, 
noted that Breese does not specifically teach "determining 
the preferred modalities of the individual'. On the other 
hand, Mizokawa teaches a system which including 
'determining the preferred modalities of the individual 
corresponds to Mizokawa's user's emotions as detailed in 
col 6, line 58-60. 

It would have been obvious to one of the ordinary 
skill in the art at the time of applicant's invention to 
incorporate the teachings of Mizokawa into modeling user's 
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emotion and personality in a computer user interface of 
Breese et al. because both are directed to user behavior 
and emotion model, more specifically Breese is directed to 
computer based capturing emotion and personality states 
that including multistage representation of emotion and 
personality variables, while Mizokawa is directed to 
controlling object using pseudo-emotions and pseudo- 
personality generated in the object, more specifically 
recognizing, updating user commands related to pseudo- 
personality, pseudo-emotions and pattern of behavior [see 
Abstract, fig 2-3]. 

One of the ordinary skill in the art at the time of 
applicant's invention to incorporate the teachings of 
Mizokawa into modeling user's emotion and personality in 
a computer interface of Breese et al. because that would 
have allowed users of Breese to use user evaluation, 
emotion recognition units to observe various pattern of 
behavior of modality, further allowing to determine pseudo- 
emotions, predetermined relationship between patterns of 
autonomous behavior, [see Mizokawa: col 2, line 15-34], 
thus improving evaluation of user's intellectual work or 
desire [col 1, line 51-60]. 

Then, on page 4, it is further stated in the final Office Action that: 

As to Claim 1 1 , Mizokawa disclosed 'preprogramming 
the device to monitor the individual for at least one specific 
type of psychomotor behavioral elements' [col 4, line 25-46]; 
'communicating an occurrence of the specific type of 
psychomotor behavioral element' [col 4, line 39-46]. 



4. Discussion of the Cited References 

Breese teaches an invention embodied in a computer user interface. 
The interface includes an observer capable of observing user behavior, an 
agent capable of conveying emotion and personality by exhibiting 
corresponding behavior to a user, and a network linking user behavior observed 
by said observer and emotion and personality conveyed by said agent. A policy 
module can dictate to the agent network desired emotion and personality states 
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to be conveyed by the agent based upon user emotion and personality states 
inferred by the observing network. Basically, Breese is designed to read 
emotional and personality characteristics of a user and broadcast emotion and 
personality to the user through an agent based on what the network deems will 
elicit a desired response from the user. As the final office action admits, Breese 
does not teach, "determining the preferred modalities of the individual." 

Mizokawa is directed towards a semi-autonomous device. The device 
senses information related to a communication from a user. That information is 
reviewed for emotional content and, while providing output related to a separate 
task, tailors the delivery of that output to be responsive to the emotional content 
sensed. 

5- Patentable Distinctions Between the Present Claims and the 
Cited References 

One of the patentable distinctions between the patents referenced and 
the claimed invention is that the references fail to teach determining the 
preferred modalities of the user. The Examiner in this case errantly suggests 
that emotions are modalities. 

Specifically, Breese in view of Mizokawa fails to at least teach, disclose, 
or suggest determining the preferred modalities of the individual. Page 4 of the 
Office Action correctly states that Breese fails to at least teach, disclose, or 
suggest determining the preferred modalities of the individual. Therefore, only 
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the suggestion that Mizokawa teaches determining the preferred modalities of 
the individual. 

As the office action states on pages 4 and 5, "Mizokawa is directed to 
controlling object using pseudo-emotions and pseudo-personality generated in 
the object, more specifically, recognizing, updating user commands related to 
pseudo-personality, pseudo-emotions, and patterns of behavior." Applicant's 
claim of determining the preferred modalities of the individual is distinctly 
different from the teachings of Mizokawa's programmed responses to sensed 
emotions. 

As detailed in the filed application, "humans have as many modalities as 
they have senses. For the purposes of a simplified discussion, we use a 
simplified web browser interface for reference. We can match the sensory 
modalities to the presentation via a grid. The grid [shown in the application] 
represents three of the many sensory modalities most humans share; visual, 
auditory and kenisthetic." Paragraph 54 of the application. This statement 
should not be read to limit the modalities to the five common senses. The 
"sensory" modalities are "input" modalities and those having ordinary skill in the 
art recognize that sensory modalities include things like visual, auditory, 
kinesthetic, olfactory, gustatory, vistubulary, proprioceptive, tactile and 
vomeronasal. There is some conjecture if the PNI is another sensory modality. 
Consciousness researchers are discovering we have more senses than we 
previously realized (vomeronasal, for example, wasn't recognized as a sense 
until the mid-1990s). It is sufficient to say that we have more sensory modalities 
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than the five a school child can name and it is reasonable to believe in the 
future we may understand we have more senses than we recognize today. 

Paragraphs 66-75 further detail modalities as claimed. As described in 
paragraphs 66 and 67, "when grid movement is determined the modality 
summations can be selected to determine if the viewer's attention is focused on 
visual, auditory, kinesthetic or other related cues. Based on the results of this 
equation, the web server can prepare in real time what the next presentation 
and interface should be in order to capture more of the viewer's attention by 
presenting the web content in modalities, which the viewer has nonconsciously 
selected." 

As described in the above passages and the cited portions of the 
application, the present invention is directed toward determining preferred 
modalities of an individual. The preferred modalities are both sensory and 
cognitive (internal?) in nature - which sensory modes does the individual prefer 
for receiving information and what internal modalities must be engaged for 
acceptance, understanding, decision making, or other actions. Some people 
would prefer to receive visually-oriented information over auditory-oriented 
information. Mizokawa does not teach identifying preferred modalities of an 
individual. Mizokawa teaches recognition of emotion and a programmable 
pseudo-emotional response. Emotions are not modalities. 

Read the passage of Mizokawa [col 6, line 55-60] the Examiner cites as 

determining the preferred modalities of the individual: 

Further, based on the presumed emotions, the user's emotion 
recognition unit A2 determines the user's current emotion. In the above, the 
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several patterns of emotions are obtained by the categorizing in advance the 
emotion expressions of the user into several patterns of emotions, such as 
joyful, sad, surprised, angry, disgusted, and fearful. 

This passage does not teach determining preferred modalities, it teaches 

categorizing emotional output. 

There are other passages of Mizokawa that teach the device tailors its 
emotions to be liked by the user, but emotions are irrelevant to modalities as 
defined in the application. This argument was raised in response to the non- 
final office action of 10/18/05 and the Examiner rebuked the argument on pages 
10-12 of the final office action, however none of the Examiner's comments 
suggested how the pseudo-emotional programming of the Mizokawa device 
amounts to "preferred modalities" as claimed in this application and as defined 
intrinsically within the application. 

As can readily be seen, the Applicant's claim is limited to determining 
preferred modalities, rendering the claim allowable in view of Breese and 
Mizokawa for at least this reason. Therefore, the Applicant respectfully 
requests submits claims 1 and 12 are allowable in that they recite features and 
steps that are neither disclosed, taught, nor suggested by Breese in view of 
Mizokawa. Furthermore, the Applicant submits that claims 2-1 1 and 13-18 are 
allowable for at least the reason that they depend either directly or indirectly 
from allowable independent claims. In re Fine , 5 U.S.P.Q. 2d 1596, 1608 (Fed. 
Cir. 1988). 
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C. Claims 2. 3. 10. 13. 16. and 17 

1. US Patent No. 5.987.415 to Breese in View of U.S. Patent No. 
6.230.111 to Mizokawa Fail to Teach. Disclose, or Suggest 
Each Element of the Claims 

On pages 5-8 of the final Office Action, claims 2, 3, 10, 13, 16, and 17 
are rejected under 35 U.S.C. §103 as allegedly being obvious over Rise in view 
of Lieberman. The limitations of Claim Group II are all directed toward 
modification of an environment of the individual whose activity is being sensed. 

2. The Examiner's Rejection 

It is stated in the final office action, on page 5, that "Mizokawa teaches a 
system which including 'modifying at least one modifiable environmental unit to 
at least partially conform to the preferred modalities' [col 6, line 46-54, fig 4]" 
with regards to claim 2. 

It is stated in the final office action, on page 5, that "Breese teaches a 
system which including 'environment unit is modified in real-time' [col 17, line 
35-43]," with regards to claim 3. 

It is stated in the final office action on page 7, that "the limitations of this 
claim have been noted in the rejection of claim 1 above. In addition, Breese 
teaches 'multi-dimensional and has a plurality of modifiable environmental units' 
[col 10, line 59-67]," with regards to claim 10. 

It is stated in the final office action, on page 6, that "Breese teaches 
algebraic transforms' [col 2, line 38-49], Breese specifically teaches Bayesian 
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network model associate with various algebraic transforms. On the other hand, 
Mizokawa teaches a system which including 'sensed psychomotor behavioral 
element is stored' [col 7, line 53-59, col 8, line 14-29], psychomotor behavioral 
element corresponds to Mizokawa's user's emotions that including obedience, 
aggressiveness, curiosity, cheerful and like as detailed in col 8, line 14-29," with 
regards to claim 13. 

It is stated in the final office action, on page 7, that "Mizokawa disclosed 
'modalities are calculated while sensing psychomotor behavioral activity and 
concurrently used for modifications to the environmental units' [col 10, line 15- 
24], with regards to claim 16." 

It is stated in the final office action, on page 8, that "Mizokawa disclosed 
'sensor includes at least one input device for a computer and the modifiable 
environmental unit includes at least one output device' [fig 1 , col 4, line 51-59]," 
with regards to claim 17. 

3. Discussion of the Cited References 

Breese teaches an invention embodied in a computer user interface. 
The interface includes an observer capable of observing user behavior, an 
agent capable of conveying emotion and personality by exhibiting 
corresponding behavior to a user, and a network linking user behavior observed 
by said observer and emotion and personality conveyed by said agent. A policy 
module can dictate to the agent network desired emotion and personality states 
to be conveyed by the agent based upon user emotion and personality states 



19 



HAYES SOLOWAY RC. 

130 W. CUSHING ST. 
TUCSON. AZ 85701 
TEL. 520.882.7623 
FAX. 520.882.7643 

175 CANAL STREET 
MANCHESTER. NH 03101 
TEL. 603.668.1400 
FAX. 603.668.8567 



inferred by the observing network. Basically, Breese is designed to read 
emotional and personality characteristics of a user and broadcast emotion and 
personality to the user through an agent based on what the network deems will 
elicit a desired response from the user. As the final office action admits, Breese 
does not teach, "determining the preferred modalities of the individual." 

Mizokawa is directed towards a semi-autonomous device. The device 
senses information related to a communication from a user. That information is 
reviewed for emotional content and, while providing output related to a separate 
task, tailors the delivery of that output to be responsive to the emotional content 
sensed. 

4. Patentable Distinctions Between the Present Claims and the 
Cited References 

This grouping of claim is joined by the fact that each are limited to 
modifying the environment relative to the calculated preferred modalities. As 
stated in Section VII. B., herein, neither of the references teaches calculating 
preferred modalities. Obviously, if the references do not teach calculating 
preferred modalities, they cannot be said to modify an environment to conform 
to those preferred modalities. However, setting aside this significant reason for 
allowance, the rejections of the Examiner fail to show the references teach the 
claimed invention. 

Claim 2 is related to modifying the environment based on the determined 
preferred modalities of the user. To reject this claim, the Examiner cites to 
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sensors in Mizokawa that perform facial and auditory analysis. These 
structures do not modify the environment of the individual. As specified in the 
cited passage, these are merely input devices. This is a clearly erroneous 
rejection. 

Claim 3 depends from claim 2 and is further limited to modifying the 
environment in real time. The Examiner cites to Breese for modifying the 
environment of the individual. However the passage cited merely updates the 
network relative to information sensed from the individual. Again, the 
environment of the user is unmodified and is not made to conform to the 
preferred modalities of the user. 

Claim 10 is rejected based on another passage of Breese. The 
Examiner blankly states the environment of Breese is multi-dimensional and has 
a plurality of modifiable environmental units. It is unclear from the passage 
what the Examiner regards as the dimensions of the environment in Breese and 
what the modifiable units are. 

Claim 13 is limited to "at least one modifiable environmental unit, 
modified by at least one instruction from the processing unit to at least partially 
conform the environment to the calculated preferred modality of the individual." 
The Examiner rejects claim 13 on page 6 of the final office action without 
identifying: 1) an environmental unit of Breese; 2) an instruction used to modify 
an environmental unit of Breese; or 3) how the environmental unit conforms to 
the preferred modality. It is unclear why this claim is rejected. 
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Claim 16 is limited to calculating modalities while sensing psychomotor 
behavior and concurrently modifying the environment. Presumably the present 
obviousness rejection is based on two references because the Examiner does 
not feel Mizokawa anticipates claim 12, yet here the Examiner is implicitly 
stating that Mizokawa teaches every element of claim 12 plus the narrower 
embodiment of claim 16. 

Claim 17 refreshingly identified a modifiable environmental unit that is an 
output device, thereby being capable of modifying the environment of an 
individual. Claim 16 and claim 17 depend from claim 13 and are allowable for 
the same reasons claim 13 is allowable. 



D. Claims 4, 5. 14. and 15 

1. US Patent No. 5,987,415 to Breese in View of U.S. Patent No. 
6,230.111 to Mizokawa Fail to Teach, Disclose, or Suggest 
Each Element of the Claims 

On pages 5-7 of the final Office Action, claims 4, 5, 14, and 15 are 
rejected under 35 U.S.C. §103 as allegedly being obvious over Breese in view 
of Mizokawa. The limitations of Claim Group III are all directed toward storing 
sensed psychomotor behavioral elements. 



HAYES SOLOWAY RC. 

130 W. CUSHING ST. 
TUCSON, AZ 85701 
TEL. 520.882.7623 
FAX. 520.882.7643 

175 CANAL STREET 
MANCHESTER, NH 03101 
TEL. 603.668.1400 
FAX. 603.668.8567 



2. The Examinees Rejection 
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It is stated in the final office action, on page 5, that "Mizokawa teaches a 
system which includes 'storing sensed psychomotor behavioral elements in a 
user history' [col 7, line 32-40, line 51-55]," in regards to claim 4. 

It is stated in the final office action, on page 6, that "Breese teaches 
algebraic transforms' [col 2, line 38-49], Breese specifically teaches Bayesian 
network model associate with various algebraic transforms. On the other hand, 
Mizokawa teaches a system which including 'sensed psychomotor behavioral 
element is stored' [col 7, line 53-59, col 8, line 14-29], psychomotor behavioral 
element corresponds to Mizokawa's user's emotions that including obedience, 
aggressiveness, curiosity, cheerful and like as detailed in col 8, line 14-29," in 
regards to claims 5 and 15. 

It is stated in the final office action, on page 7, that "Mizokawa disclosed 
'memory device to store sensed psychomotor behavioral activity of the 
individual' [col 7, line 60-64]," in regards to claim 14. 
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3. Discussion of the Cited References 

Breese teaches an invention embodied in a computer user interface. 
The interface includes an observer capable of observing user behavior, an 
agent capable of conveying emotion and personality by exhibiting 
corresponding behavior to a user, and a network linking user behavior observed 
by said observer and emotion and personality conveyed by said agent. A policy 
module can dictate to the agent network desired emotion and personality states 
to be conveyed by the agent based upon user emotion and personality states 
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inferred by the observing network. Basically, Breese is designed to read 
emotional and personality characteristics of a user and broadcast emotion and 
personality to the user through an agent based on what the network deems will 
elicit a desired response from the user. As the final office action admits, Breese 
does not teach, "determining the preferred modalities of the individual." 

Mizokawa is directed towards a semi-autonomous device. The device 
senses information related to a communication from a user. That information is 
reviewed for emotional content and, while providing output related to a separate 
task, tailors the delivery of that output to be responsive to the emotional content 
sensed. 

4. Patentable Distinctions Between the Present Claims and the 
Cited References 

a. Storing sensed psychomotor behavioral element 

The Examiner does not suggest that Mizokawa teaches sensing 
psychomotor behavioral elements. It is difficult to conceive how Mizokawa can 
store sensed information without also sensing the information. 
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b. Linear algebraic transforms 

The Examiner has repeatedly, during the course of this prosecution, 
rejected claim 5 based on finding various references storing information through 
the use of algebraic transforms. In all cases the references store information 
using algebraic transforms. In none of the cases do the references use linear 
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algebraic transforms. The latest reference, Breese, teaches storing information 
through the use of algebraic transforms, as the Examiner indicates on pages 6 
and 12 of the final Office Action. Breese does not teach storing information 
using linear algebraic transforms, which is the claim limitation. 

E. Claims 6-8 

1. US Patent No. 5.987.415 to Breese in View of U.S. Patent No. 
6.230.111 to Mizokawa Fail to Teach. Disclose, or Suggest 
Each Element of the Claims 

On pages 6 and 7 of the final Office Action, claims 6-8 are rejected under 
35 U.S.C. §103 as allegedly being obvious over Breese in view of Mizokawa. 
The limitations of Claim Group IV are all directed toward determining a preferred 
combination of modalities and an ordering thereof. 

2. The Examiner's Rejection 

It is stated in the final office action, on page 6, that "Mizokawa teaches 
determining preferred modalities including determining a preferred combination 
of modalities and an ordering of modalitites by preference thereby further 
defining a focus of the individual's attention' [col 7, line 11-17, line 21-24]," with 
respect to claim 6. 

It is stated in the final office action, on page 6, that "Mizokawa disclosed 
'modifying the environmental unit to provide content in the environment in the 
preferred combination of modalities and the order of the modalities by 
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preference whereby the combination and the order are placed in at least one 
respective co-ordinate group of representational geometry to which attention of 
the individual is drawn, as indicated by the psychomotor behavioral element' 
[col 6, line 46-62]," with respect to claim 7. 

It is stated in the final office action, on page 7, that "Breese disclosed 
'defining a psychodynamic and a cognitive behavioral model using preferred 
combination modalities and the order of modalities' [col 5, line 5-10, line 23-28]; 
'modifying at least one environmental unit as a function of the psychodynamic 
behavioral model and the cognitive behavioral model' [col 8, line 35-43]," with 
respect to claim 8. 

3. Discussion of the Cited References 

Breese teaches an invention embodied in a computer user interface. 
The interface includes an observer capable of observing user behavior, an 
agent capable of conveying emotion and personality by exhibiting 
corresponding behavior to a user, and a network linking user behavior observed 
by said observer and emotion and personality conveyed by said agent. A policy 
module can dictate to the agent network desired emotion and personality states 
to be conveyed by the agent based upon user emotion and personality states 
inferred by the observing network. Basically, Breese is designed to read 
emotional and personality characteristics of a user and broadcast emotion and 
personality to the user through an agent based on what the network deems will 
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elicit a desired response from the user. As the final office action admits, Breese 
does not teach, "determining the preferred modalities of the individual." 

Mizokawa is directed towards a semi-autonomous device. The device 
senses information related to a communication from a user. That information is 
reviewed for emotional content and, while providing output related to a separate 
task, tailors the delivery of that output to be responsive to the emotional content 
sensed. 
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4. Patentable Distinctions Between the Present Claims and the 
Cited References 

a. Determining a preferred combination of modalities 

On page 4 of the final Office Action, the Examiner states, "preferred 
modalities of the individual corresponds to Mizokawa's user's emotions," a point 
contested by the Applicant above. However, in rejecting claim 6, the Examiner 
does not cite to a section of the reference that purports to combine Mizokawa's 
user's emotions or determine a preferred combination of Mizokawa's user's 
emotions. It would appear the Examiner's definition of "preferred modalities" 
has changed for this rejection. 

In paragraph 80 of the application, preferred combinations of modalities 
is explained: 

Another narrow embodiment of the inventive method 1 0 
narrows the step of determining preferred modalities to further 
determine 20 a preferred combination of modalities and 22 an 
ordering of modalities by preference thereby further defining a 
focus of the individual's preferred modalities. The present 
inventive method 10 may determine 14 that the individual 



27 



HAYES SOLOWAY RC. 

130 W. CUSHING ST. 
TUCSON. AZ 85701 
TEL. 520.882.7623 
FAX. 520.882.7643 

175 CANAL STREET 
MANCHESTER, NH 03101 
TEL. 603.668.1400 
FAX. 603.668.8567 



prefers to receive information audially as opposed to visually. 
This narrower embodiment would further determine 20 the 
individual prefers to receive information audially 65% and 
visually 35%. 

It is fairly evident that Mizokawa does not teach a preferred combination of 
modalities as it is explained in the application. 

On page 12 of the final Office Action, the Examiner declares the cited 
passage in column 7 teaches "establishing relationship between emotions and 
characteristics of sounds/voices and facial expression." Again, it sounds like 
this passage is determining "user's emotions" not determining "preferred 
combinations of user's emotions" and certainly not doing anything related to 
determining preferred combinations of modalities, as defined in the application. 

b. Determining ... an ordering of modalities by preference 

The Examiner chose to find the limitation of "ordering" in the same 

paragraph as the "combination" rejection above. In paragraph 80 of the 

application, following the quoted passage above, the Applicant further explained 

the ordering limitation with the following example: 

This narrower embodiment would further determine 22 
the individual prefers to receive information in the 
aforementioned percentages by receiving 50% of the visual 
information first and then receiving 100% of the audiol 
information, before finishing the information presentation with 
the remaining 50% of the visual information. 

Nothing in Breese and Mizokawa can be considered to order preferred 

modalities by preference, as claimed. Even stretching the definition of preferred 



28 



modalities to user emotions, as the Examiner has done, it is difficult to conceive 
of an ordering of emotions as claimed. 

c. Defining a focus of the individual's attention 

In marked contrast to Breese and Mizokawa, the claimed invention 
defines where the individual's attention is focused. The claimed invention 
defines where the individual's attention is focused by determining a preferred 
combination of modalities and an ordering of modalities by preference for the 
individual. For instance, the claimed invention may determine that an 
individual's preferred mode of operation is to receive information audibly and 
kinesthetically (preferred combination of modalities) and that the individual has 
a greater preference toward audible information (ordering by preference), 
thereby determining the focus of the individual is on what he/she is hearing 
primarily and on what he/she is feeling secondarily. 

The claimed invention identifies/observes where the focus of the person's 
attention is by analyzing the preferred modalities of the individual. This "thereby 
clause" operates to give meaning to the limitations in claim 6. Neither Breese, 
nor Mizokawa can be said to identify/observe where the focus of the person's 
attention is by analyzing the preferred modalities of the individual. Neither 
Breese, nor Mizokawa teach or disclose all of the elements of claim 6. 
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F. Claim 9 

1. US Patent No. 5.987.415 to Breese in View of U.S. Patent No. 
6.230.111 to Mizokawa Fail to Teach. Disclose, or Suggest 
Each Element of the Claim 

On page 8 of the final Office Action, claim 9 is rejected under 35 U.S.C. 
§103 as allegedly being obvious over Breese in view of Mizokawa. The 
limitation of Claim Group V is directed toward an equation for calculating the 
combination and order of modalities. 

2. The Examiner's Rejection 

It is stated in the final Office Action, on page 8, that "both Breese, 
Mizokawa do not teach 'modalities calculation by an equation', although 
Breese teaches modeling a user's emotion and personality specifically using 
Bayesian network inference algorithm [see abstract, col 12, line 29-35], while 
Mizokawa teaches control system for controlling object using pseudo-emotions 
and personally specifically calculating using rules or functional equations for 
pseudo emotions levels and personalities [col 10, line 15-18]. Therefore, it 
would have been obvious to one of the ordinary skill in the art at the time of 
applicant's invention to not only sense emotions, but also establish relationship 
between emotions, environmental information to evaluate the behavioral output 
as suggested by Mizokawa [col 9, line 16-21]," with respect to claim 9. 
(emphasis added) 
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3. Discussion of the Cited References 

Breese teaches an invention embodied in a computer user interface. 
The interface includes an observer capable of observing user behavior, an 
agent capable of conveying emotion and personality by exhibiting 
corresponding behavior to a user, and a network linking user behavior observed 
by said observer and emotion and personality conveyed by said agent. A policy 
module can dictate to the agent network desired emotion and personality states 
to be conveyed by the agent based upon user emotion and personality states 
inferred by the observing network. Basically, Breese is designed to read 
emotional and personality characteristics of a user and broadcast emotion and 
personality to the user through an agent based on what the network deems will 
elicit a desired response from the user. As the final office action admits, Breese 
does not teach, "determining the preferred modalities of the individual." 

Mizokawa is directed towards a semi-autonomous device. The device 
senses information related to a communication from a user. That information is 
reviewed for emotional content and, while providing output related to a separate 
task, tailors the delivery of that output to be responsive to the emotional content 
sensed. 

4. Patentable Distinctions Between the Present Claims and the 
Cited References 

a. The combination and order of modalities is calculated 

The Examiner admits specifically that neither reference uses an equation 
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to calculate modalities. Here, again, the Examiner misstates the limitation. 
Claim 9 is drawn to calculating a combination and order of preferred modalities. 
The Examiner explicitly states that Breese fails to teach determining preferred 
modalities, so how Breese could be interpreted to contribute to calculating 
modalities, a combination thereof, and/or an order thereof is not explainable. 

It is unclear how the Examiner believes Mizokawa teaches the claim 
limitation. The claim states the combination and order of modalities is 
calculated by a given equation. The Examiner states on page 8 of the final 
Office Action, "it would have been obvious to one of ordinary skill in the art at 
the time of applicant's invention to ... establish relationship between emotions, 
environmental information to evaluate behavioral output." The Applicant has no 
idea how the latter, if true, could be said to teach or disclose the former. 
Further, it may have been obvious to want to establish that relationship, but it is 
unclear how the relationship would have been obvious. The Examiner appears 
to have failed to allege a case of prima facie obviousness. 

The Examiner's statement on page 14 is an effort to shed light on the 
rejection, but it is still confusing. According to the Examiner, Breese teaches 
Bayesian inference algorithms based on user's emotions and Mizokawa 
suggests determining pseudo-emotions and, while neither reference teaches 
"modalities calculation by an equation", combining these references teaches the 
limitations of claim 9. Neither the word "combination" nor the word "order" 
appear in either of the Examiner's explanations for rejecting claim 9, yet 
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"environmental information" and "pseudo-personality" appear repeatedly, 
although it is unclear how that relates to claim 9. 

b. An equation: £f ^f(£G .(8Xi/ 8it))/(£Gi(dXi/dti)))dGidti qc 

y(GV 

Claim 9 is limited to a specific equation for calculating a combination and 
order of modalities. Neither Breese, nor Mizokawa teaches this equation. As 
the Examiner stated, neither reference show "modalities calculation by an 
equation." It is unclear why this specific equation is obvious in any way. 

G. EXAMINER INTERVIEW 

An Examiner Interview was held telephonically on March 23, 2005, at 
2PM. Examiner Channavajjala, Attorney Todd Sullivan, Patent Agent Amit 
Singh, and the inventor, Joseph Carrabis participated in the interview. The 
Applicant scheduled the interview, in part, in an attempt to explain to the 
Examiner the difference between neuromotor elements of activity and 
psychomotor elements of activity as well as attempting to better understand the 
Examiner's interpretation of the claims in an effort to advance the prosecution of 
the present case. During the interview, the Examiner was unwilling to discuss 
the Rise and Lieberman references, he resisted and, ultimately, avoided viewing 
an online demonstration of the invention intended to allow him to better 
appreciate the invention, and he was unable to view with the Applicant an 



33 



HAYES SOLOWAY RC. 

130 W. CUSHING ST. 
TUCSON, AZ 85701 
TEL. 520.882.7623 
FAX. 520.882.7643 

175 CANAL STREET 
MANCHESTER, NH 03101 
TEL. 603.668.1400 
FAX. 603.668.8567 



Interview Agenda that had been entered into the PAIRS system more than one 
week prior to the interview. 

More to the point, the Examiner used the Examiner Interview as an 
opportunity to voice other reasons he was unable to allow the claims, which 
reasons have not appeared in any written office actions. As a result, while the 
following arguments have not been presented in a written office action, they 
have been encountered in the Examiner Interview and are addressed herein. 

On multiple occasions during the interview, Examiner Channavajjala 
expressed an unwillingness to allow the claims as written because he was of 
the opinion the claims were too broad. Breadth of the claims, without more, is 
not grounds for rejecting or refusing to allow claims. As laid out in Section 
2173.04 of the MPEP, undue breadth of the claim may be addressed in several 
ways, including: arguing that it is too broad because it is unsupported by the 
enabling disclosure under 35 U.S.C. 112; arguing that the claims are so broad 
they fail to set out that which the inventor claims as the invention under 35 
U.S.C. 112; and arguing the claims are too broad because they read on the 
prior art under 35 U.S.C. 102 and 103. The simple fact that the Examiner feels 
the claims are too broad is not sufficient reason to withhold allowance of the 
claims. 

The Examiner mentioned during the interview that one of the reasons the 
USPTO cannot allow claims that are too broad is that overly broad patent claims 
may have the unintended consequence of stifling innovation and turning public 
opinion against the USPTO. While this opinion is a common concern in the 
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world of patents, particularly with regards to national and international concerns 
regarding software patents, public policy concerns are generally not grounds for 
withholding allowance of patent claims. The only public policy basis, as 
identified in the MPEP, that may be used as grounds for a rejection is under 35 
U.S.C. 101 for lack of utility. USPTO examiners have more than enough work 
to do without attempting to determine how claims, once allowed, may affect 
industry. 

The Examiner further suggested that if claims 6, 7, and 9 were combined 
into claim 1 , he may allow that claim. This suggestion is indicative of the desire 
to get more limitations into the independent claim for the sake of making the 
claim less broad. The fact that both claim 7 and claim 9 include the limitations 
of claims 1 and 6 and that claim 7 is rejected based on the same references 
used to reject claim 9 make this suggestion confounding. Examiner 
Channavajjala was unable to explain why he believed the combination of these 
limitations may overcome the teachings of the prior art. Claim allowance should 
not be based on word count. 
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CLAIMS APPENDIX 

1 . A method of obtaining information regarding an environment for an individual, 
having preferred modalities and engaged in activity, using a programmable 
device, said method comprising the steps of: 

sensing at least one psychomotor behavioral element of the activity 
engaged by the individual; and 

determining the preferred modalities of the individual based on the 
psychomotor behavioral element of the activity engaged by the individual. 



2. The method of Claim 1 further comprising modifying at least one modifiable 
environmental unit to at least partially conform to the preferred modalities. 



3. The method of Claim 2 wherein the environment unit is modified in real-time. 



4. The method of Claim 1 further comprising storing the sensed psychomotor 
behavioral element in a user history. 
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5. The method of Claim 4 wherein the sensed psychomotor behavioral element 
is stored in terms of preferred representational geometries via linear algebraic 
transforms. 

6. The method of Claim 1 wherein the step of determining preferred modalities 
includes determining a preferred combination of modalities and an ordering of 
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modalities by preference thereby further defining a focus of the individual's 
attention. 

7. The method of Claim 6 further comprising the step of modifying the 
environmental unit to provide content in the environment in the preferred 
combination of modalities and the order of modalities by preference whereby 
the combination and the order are placed in at least one respective co-ordinate 
group of representational geometry to which attention of the individual is drawn, 
as indicated by the psychomotor behavioral element. 

8. The method of Claim 6 further comprising: 

defining a psychodynamic and a cognitive behavioral model using the 
preferred combination of modalities and the order of modalities; and 

modifying at least one environmental unit as a function of the 
psychodynamic behavioral model and the cognitive behavioral model. 

9. The method of Claim 6 wherein the combination and order of modalities is 
calculated by an equation: zf ^((ZGi(8Xi/8it))/(ZGi(dXi/dti)))dGidti <x ¥(G) 

10. The method of Claim 1 wherein the environment is multi-dimensional and 
has a plurality of modifiable environmental units. 

1 1 . The method of Claim 1 further comprising 
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preprogramming the device to monitor the individual for at least one 
specific types of psychomotor behavioral elements; and 

communicating an occurrence of the specific type of psychomotor 
behavioral element. 

12. A programmable apparatus for obtaining information regarding an 
environment to an individual having preferred modalities, said apparatus 
comprising: 

at least one sensor for sensing psychomotor behavioral activity of the 
individual; and 

a processing unit connected to the sensor for receiving the sensed 
psychomotor behavioral activity and calculating the individual's preferred 
modalities based on the sensed psychomotor behavioral activity. 



13. The apparatus of Claim 12 further comprising at least one modifiable 
environmental unit, modified by at least one instruction from the processing unit 
to at least partially conform the environment to the calculated preferred modality 
of the individual. 
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14. The apparatus of Claim 12 further comprising a memory device to store 
sensed psychomotor behavioral activity of the individual. 
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15. The apparatus of Claim 14 wherein the processing unit uses stored sensed 
psychomotor behavioral activity of the individual to refine the preferred modality 
calculation. 

16. The apparatus of Claim 13 wherein the preferred modalities are calculated 
while sensing psychomotor behavioral activity and concurrently used for 
modifications to the environmental units. 



17. The apparatus of Claim 13 wherein the sensor includes at least one input 
device for a computer and the modifiable environmental unit includes at least 
one output device. 



18. The apparatus of Claim 12 further comprising an indicator connected to the 
processing unit, wherein the processing unit is preprogrammed to monitor for 
specific psychomotor behavioral activity and the indicator indicates at least one 
of the group consisting of: 

a match between the sensed psychomotor behavioral activity and the 
specific psychomotor behavioral activity; and 

a nonmatch between the sensed psychomotor behavioral activity and the 
specific psychomotor behavioral activity. 
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EVIDENCE APPENDIX 

None submitted by the Applicant and none entered by the Examiner. 
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RELATED PROCEEDINGS APPENDIX 

No related proceedings exist or existed. 
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NCLUSION 



In view of the foregoing, it is respectfully requested that the Examiner's 
rejection of the subject application be reversed. 

Respectfully submitted, 




Todd A. Sullivan 
Attorney for Appellant 
Reg. No. 47,117 



CERTIFICATE OF MAILING 



I hereby certify that this correspondence is being deposited with the 
United States Postal Service as First Class Mail in an envelope addressed to: 
Mail Stop Appeal Brief - Patents, Commissioner for Patents, PO Box 1450, 

Alexandria, VA 22313-1450 on .Jjx&u n, Zcxyu at Manchester, 

New Hampshire. J 

By: ^/n^^Vfue^ 

7 Kristine Stevens 



HAYES SOLOWAY PC. 

130 W. CUSHING ST. 
TUCSON, AZ 85701 
TEL. 520.882.7623 
FAX. 520.882.7643 



175 CANAL STREET 
MANCHESTER, NH 03101 
TEL. 603.668.1400 
FAX. 603.668.8567 



42 



